Gaze-tracking system and method of tracking user&#39;s gaze

ABSTRACT

A gaze-tracking system for use in a head-mounted display apparatus. The gaze-tracking system includes at least one illuminator for emitting light pulses; at least one first optical element comprising a plurality of micro-to-nano-sized components, shaped and arranged relative to each other in a manner that, when incident thereupon, a structure of the light pulses is modified to produce structured light, wherein the produced structured light is used to illuminate a user&#39;s eye; at least one camera for capturing an image of reflections of the structured light from the user&#39;s eye, wherein the image is representative of a form and a position of the reflections on an image plane of the at least one camera; and a processor configured to control the at least one illuminator and the at least one camera, and to process the captured image to detect a gaze direction of the user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 15/648,886, titled “GAZE-TRACKING SYSTEM AND METHOD OF TRACKINGUSER'S GAZE” and filed on Jul. 13, 2017, which is incorporated herein byreference. Furthermore, the U.S. patent application Ser. No. 15/648,886is a continuation-in-part of U.S. patent application Ser. No.15/366,424, titled “DISPLAY APPARATUS AND METHOD OF DISPLAYING USINGFOCUS AND CONTEXT DISPLAYS” and filed on Dec. 1, 2016, which is alsoincorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates generally to display apparatuses; andmore specifically, to gaze-tracking systems for use in head-mounteddisplay apparatuses, such gaze-tracking systems comprising illuminators,optical elements, cameras and processors. Furthermore, the presentdisclosure also relates to methods of tracking a user's gaze via theaforementioned gaze-tracking systems.

BACKGROUND

In recent times, there has been a rapid increase in use of technologiessuch as virtual reality, augmented reality, and so forth, for presentinga simulated environment (or a simulated world) to a user. Typically, theuser uses a device (for example, such as a virtual reality device, anaugmented reality device, and the like) for experiencing such asimulated environment. Furthermore, in use, the user generally wears(namely, supports) the device on his/her head.

Nowadays, such devices often employ a technique such as gaze-tracking(namely, eye tracking) to determine a gaze direction of the user.Typically, the gaze-tracking is associated with determination ofposition of pupils of the user's eyes. Generally, an illumination sourceis employed for emitting light towards the user's eyes, and a camera isemployed for capturing an image depicting the pupils of the user's eyesand reflection(s) of the emitted light from the user's eyes.Furthermore, reflections of the emitted light from the user's eyes areused as reference for determining the position of the pupils of theuser's eyes with respect to the reflections.

However, there exist a number of limitations associated withimplementations of the aforementioned gaze-tracking techniques. Firstly,a portion of the reflections may be occluded when the user's eyes arepartially closed. In such an instance, some of the reflections may beabsent (namely, some light may not be reflected by the surface of theuser's eyes). Consequently, such occlusion of the reflections leads toinaccurate determination of the position of the pupils of the user'seyes. Secondly, since some of the reflections may be obscured by eyelidsof the user, positions of the visible reflections may be inaccuratelyidentified. Such inaccurate identification of the position of thevisible reflections further leads to inaccuracies in gaze detection.Thirdly, various ambient light sources may be present near the user thatmay emit light towards the user's eyes. In such an instance, reflectionsof ambient light may be inaccurately considered to be reflections of thelight emitted by the plurality of illuminators. Consequently, in such acase, the position of the pupils of the user's eyes is inaccuratelydetermined. Fourthly, existing gaze-tracking techniques do notcompensate for changes in pupil geometry (namely, on account of pupilcontraction and pupil dilation). Consequently, additional inaccuracies(for example, such as geometric aberrations, reflection artefacts, andthe like) are introduced whilst tracking the user's gaze.

Therefore, in light of the foregoing discussion, there exists a need toovercome the aforementioned drawbacks associated with conventionalgaze-tracking techniques.

SUMMARY

The present disclosure seeks to provide a gaze-tracking system for usein a head-mounted display apparatus.

The present disclosure also seeks to provide a method of tracking auser's gaze, via a gaze-tracking system of a head-mounted displayapparatus.

The present disclosure seeks to provide a solution to the existingproblem of inaccuracies in existing gaze-tracking techniques due toocclusion of some of multiple reflections. An aim of the presentdisclosure is to provide a solution that overcomes at least partiallythe problems encountered in the prior art, and provides a robust andefficient gaze-tracking system that eliminates inaccuracies associatedwith use of existing gaze-tracking techniques.

In one aspect, an embodiment of the present disclosure provides agaze-tracking system for use in a head-mounted display apparatus, thegaze-tracking system comprising:

at least one illuminator for emitting light pulses;

at least one first optical element comprising a plurality ofmicro-to-nano-sized components, the plurality of micro-to-nano-sizedcomponents being shaped and arranged relative to each other in a mannerthat, when incident upon the plurality of micro-to-nano-sizedcomponents, a structure of the light pulses is modified to producestructured light, wherein the produced structured light is to be used toilluminate a user's eye when the head-mounted display apparatus is wornby the user;

at least one camera for capturing an image of reflections of thestructured light from the user's eye, wherein the image isrepresentative of a form of the reflections and a position of thereflections on an image plane of the at least one camera; and

a processor coupled in communication with the at least one illuminatorand the at least one camera, wherein the processor is configured tocontrol the at least one illuminator and the at least one camera, and toprocess the captured image to detect a gaze direction of the user.

In another aspect, an embodiment of the present disclosure provides amethod of tracking a user's gaze, via a gaze-tracking system of ahead-mounted display apparatus, the method comprising:

producing structured light, via at least one illuminator and at leastone first optical element of the gaze-tracking system, to illuminate auser's eye when the head-mounted display apparatus is worn by the user,wherein the at least one first optical element comprises a plurality ofmicro-to-nano-sized components that are shaped and arranged relative toeach other in a manner that, when incident upon the plurality ofmicro-to-nano-sized components, a structure of light pulses emitted bythe at least one illuminator is modified to produce the structuredlight;

capturing an image of reflections of the structured light from theuser's eye, via at least one camera of the gaze-tracking system, whereinthe image is representative of a form of the reflections and a positionof the reflections on an image plane of the at least one camera; and

processing the captured image to detect a gaze direction of the user.

Embodiments of the present disclosure substantially eliminate or atleast partially addresses the aforementioned problems in the prior art,and enables enable accurate and efficient tracking of the user's gaze.

Additional aspects, advantages, features and objects of the presentdisclosure would be made apparent from the drawings and the detaileddescription of the illustrative embodiments construed in conjunctionwith the appended claims that follow.

It will be appreciated that features of the present disclosure aresusceptible to being combined in various combinations without departingfrom the scope of the present disclosure as defined by the appendedclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

The summary above, as well as the following detailed description ofillustrative embodiments, is better understood when read in conjunctionwith the appended drawings. For the purpose of illustrating the presentdisclosure, exemplary constructions of the disclosure are shown in thedrawings. However, the present disclosure is not limited to specificmethods and instrumentalities disclosed herein. Moreover, those in theart will understand that the drawings are not to scale. Whereverpossible, like elements have been indicated by identical numbers.

Embodiments of the present disclosure will now be described, by way ofexample only, with reference to the following diagrams wherein:

FIG. 1 illustrates a block diagram of a gaze-tracking system for use ina head-mounted display apparatus, in accordance with an embodiment ofthe present disclosure;

FIGS. 2A-2F illustrate exemplary implementations of a gaze-trackingsystem in use within a head-mounted display apparatus, in accordancewith various embodiments of the present disclosure;

FIG. 3 illustrates an exemplary image of a user's eye captured by atleast one camera of a gaze-tracking system, in accordance with anembodiment of the present disclosure;

FIGS. 4A and 4B illustrate zoomed-in views of exemplary images of auser's eye depicting a plurality of glints therein, in accordance withdifferent embodiments of the present disclosure; and

FIG. 5 illustrates steps of a method of tracking a user's gaze, via agaze-tracking system of a head-mounted display apparatus, in accordancewith an embodiment of the present disclosure.

In the accompanying drawings, an underlined number is employed torepresent an item over which the underlined number is positioned or anitem to which the underlined number is adjacent. A non-underlined numberrelates to an item identified by a line linking the non-underlinednumber to the item. When a number is non-underlined and accompanied byan associated arrow, the non-underlined number is used to identify ageneral item at which the arrow is pointing.

DETAILED DESCRIPTION OF EMBODIMENTS

The following detailed description illustrates embodiments of thepresent disclosure and ways in which they can be implemented. Althoughsome modes of carrying out the present disclosure have been disclosed,those skilled in the art would recognize that other embodiments forcarrying out or practising the present disclosure are also possible.

In one aspect, an embodiment of the present disclosure provides agaze-tracking system for use in a head-mounted display apparatus, thegaze-tracking system comprising:

at least one illuminator for emitting light pulses;

at least one first optical element comprising a plurality ofmicro-to-nano-sized components, the plurality of micro-to-nano-sizedcomponents being shaped and arranged relative to each other in a mannerthat, when incident upon the plurality of micro-to-nano-sizedcomponents, a structure of the light pulses is modified to producestructured light, wherein the produced structured light is to be used toilluminate a user's eye when the head-mounted display apparatus is wornby the user;

at least one camera for capturing an image of reflections of thestructured light from the user's eye, wherein the image isrepresentative of a form of the reflections and a position of thereflections on an image plane of the at least one camera; and

a processor coupled in communication with the at least one illuminatorand the at least one camera, wherein the processor is configured tocontrol the at least one illuminator and the at least one camera, and toprocess the captured image to detect a gaze direction of the user.

In another aspect, an embodiment of the present disclosure provides amethod of tracking a user's gaze, via a gaze-tracking system of ahead-mounted display apparatus, the method comprising:

producing structured light, via at least one illuminator and at leastone first optical element of the gaze-tracking system, to illuminate auser's eye when the head-mounted display apparatus is worn by the user,wherein the at least one first optical element comprises a plurality ofmicro-to-nano-sized components that are shaped and arranged relative toeach other in a manner that, when incident upon the plurality ofmicro-to-nano-sized components, a structure of light pulses emitted bythe at least one illuminator is modified to produce the structuredlight;

capturing an image of reflections of the structured light from theuser's eye, via at least one camera of the gaze-tracking system, whereinthe image is representative of a form of the reflections and a positionof the reflections on an image plane of the at least one camera; and

processing the captured image to detect a gaze direction of the user.

The aforementioned gaze-tracking system and the method of tracking auser's gaze employ the plurality of micro-to-nano-sized components andthe at least one illuminator, to illuminate the user's eye with thestructured light when the head-mounted display apparatus is worn by theuser. Such use of the structured light enables to accurately determinethe positions of reflections of the structured light based on a formthereof. Consequently the detected gaze direction by using suchstructured light is highly accurate. Additionally, the use of thestructured light allows for substantially overcoming errors associatedwith occlusion of reflections of light. Furthermore, errors associatedwith presence of reflections from ambient light sources aresubstantially minimized within the described gaze-tracking system.Moreover, the gaze-tracking system accommodates changes in pupilgeometry whilst tracking the user's gaze. Beneficially, theaforementioned gaze-tracking system allows for determining anorientation of the user's eye. The described method is efficient, robustand reliable.

Throughout the present disclosure, the term “head-mounted displayapparatus” used herein relates to specialized equipment that isconfigured to display an input image to the user thereof when thehead-mounted display apparatus is worn by the user on his/her head. Insuch an instance, the head-mounted display apparatus is operable to actas a device (for example, such as a virtual reality headset, anaugmented reality headset, a pair of virtual reality glasses, a pair ofand augmented reality glasses, and so forth) for presenting the inputimage to the user.

Throughout the present disclosure, the term “gaze-tracking system” usedherein relates to specialized equipment for detecting a direction ofgaze (namely, the gaze direction) of the user. The head-mounted displayapparatus uses the gaze-tracking system for determining the aforesaidgaze direction via non-invasive techniques. Beneficially, an accuratedetection of the gaze direction facilitates the head-mounted displayapparatus to closely implement gaze contingency thereon. As an example,the gaze-tracking system may be employed to detect the gaze direction ofthe user's eye for projecting (i) a region of a visual scene whereat theuser's gaze is focused, on and around the fovea of the user's eye, and(ii) a remaining region of the visual scene on the retina of the user'seye, of which the fovea is just a small part. Therefore, even upon achange in the gaze direction (namely, due to a movement of the user'seye), active foveation is implemented within the head-mounted displayapparatus.

It is to be understood that the gaze-tracking system may also bereferred to as an “eye-tracker system”, a “means for detecting a gazedirection”, a “means for tracking a gaze direction”, or a “gaze-trackingunit”.

Throughout the present disclosure, the term “at least one illuminator”used herein relates to at least one light source configured to emit thelight pulses for illuminating the user's eye. The at least oneilluminator is employed to emit the light pulses to illuminate theuser's eye when the head-mounted display apparatus is worn by the user.Optionally, the at least one illuminator could emit the light pulsesperiodically or intermittently. Alternatively, optionally, the at leastone illuminator emits light continuously. It will be appreciated thatthe light pulses emitted by the at least one illuminator are reflectedfrom an outer surface (for example, such as cornea) of the user's eye,thereby constituting corneal reflections (namely, glints) in the user'seye.

Optionally the light pulses emitted by the at least one illuminator havean infrared wavelength or a near-infrared wavelength. The light pulsesof infrared or near-infrared wavelength are invisible to the human eye,thereby, reducing unwanted distraction when such light pulses areincident upon the user's eye. Alternatively, optionally, the lightpulses emitted by the at least one illuminator have a visiblewavelength.

Optionally, the at least one illuminator is implemented by way of atleast one of: infrared light-emitting diodes, infrared lasers, infraredlight projectors, infrared light-emitting diode based displays, visiblelight-emitting diodes, visible light lasers, visible light projectors.

In an embodiment, the at least one illuminator is operable to illuminateone eye of the user. Optionally, in such a case, the at least oneilluminator comprises at least two illuminators, wherein there is atleast one illuminator per eye. In another embodiment, the at least oneilluminator is operable to illuminate both eyes of the user.

Optionally, an intensity of the light pulses emitted by the at least oneilluminator is adjustable. Optionally, in this regard, the processor isconfigured to control the at least one illuminator to adjust theintensity of light pulses emitted thereby.

Throughout the present disclosure, the term “at least one first opticalelement” used herein relates to an optical component that is configuredto alter optical properties of the emitted light pulses. Additionally,optionally, the at least one first optical element is configured toalter an optical path of the emitted light pulses. The at least onefirst optical element is arranged on the optical path of the emittedlight pulses, namely, between the at least one illuminator and theuser's eye.

Optionally, the at least one first optical element is implemented by wayof at least one: a flexible transparent film, flexible transparent foil,a semi-transparent mirror, a fully reflective mirror, a prism, apolarizer, an optical waveguide, a beam splitter (namely, a halfmirror), a glass plate, a lens.

Optionally, in operation, the at least one illuminator emits the lightpulses in a direction that is substantially along a view direction ofthe user's eye. Optionally, in such an instance, the at least one firstoptical element is configured to pass the emitted light pulsestherethrough.

Alternatively, optionally, in operation, the at least one illuminatoremits the light pulses in a direction that is at a predefined angle to aview direction of the user's eye. In this regard, the emitted lightpulses are incident upon the at least one first optical element and theat least one first optical element is configured to direct the emittedlight pulses towards the user's eye via at least one of: reflection ofthe emitted light pulses, refraction of the emitted light pulses. As anexample, the light can be emitted by the at least one illuminator in adirection that is substantially perpendicular to a view direction of theuser's eye. In such an example, the at least one first optical elementcould be arranged in a manner that it reflects the emitted light pulsestowards the user's eye.

Optionally, a size of the plurality of micro-to-nano-sized componentsranges between 1 nanometer (nm) and 10 micrometer (Um). More optionally,the size of the plurality of micro-to-nano-sized components rangesbetween 1 nm and 100 nm. Yet more optionally, the size of the pluralityof micro-to-nano-sized components ranges between 10 and 50 nm.

Optionally, the plurality of micro-to-nano-sized components are made ofat least one of: polymethyl methacrylate (PMMA), polycarbonate (PC),polystyrol (PS), cyclo olefin polymer (COP) and/or cycloolefin-copolymer (COC).

Throughout the present disclosure, the term “structured light” refers tolight that is emitted onto a surface (such as the cornea of the user'seye) in a predefined pattern, such as a matrix or a grid. Furthermore,the structured light is produced by employing the at least oneilluminator and the at least one first optical element. In an example,the structured light is produced in a pattern, for example, such aslinear, circular, triangular, rectangular, concentric circular (such as,circles having decreasing or increasing diameters with respect to eachother and having a common center) and so forth. In another example, thestructured light is produced in a predefined pattern comprising text(such as one or more alphabets), symbols (such as symbol for Greekletter omega (Ω)), designs (such as logos) and so forth.

Optionally, when the structured light is incident upon the surface ofthe user's eye, its reflections appear in a form of a plurality ofglints, wherein the plurality of glints have a predefined shape.Optionally, the predefined shape has a pattern comprising six charactersarranged into two columns of three characters each. Alternatively,optionally, the predefined shape has a pattern comprising ninecharacters arranged into three columns of three characters each. Moreoptionally, the characters are substantially V-shaped. Optionally, amiddle character of a given column is oriented differently from othercharacters of the given column. In this regard, the plurality ofmicro-to-nano-sized components are arranged in a manner such that astructure of the emitted light pulses incident thereupon is modified toproduce the plurality of glints having the aforesaid pattern. In anexample, if a first and a third character of the given column is “>”,then a second character (namely, the middle character) of the givencolumn may be “<”. Beneficially, such a pattern of the plurality ofglints provides a unique, distinguishable shape of the structured light.Therefore, it is easier to distinguish between the structured light andunwanted light from ambient light sources. Optionally, the shape of theplurality of glints comprises characters having a symmetrical shapeabout only one axis. Notably, a shape of such characters only repeatswhen such characters are rotated through 360 degrees, thereby, allowingfor rotation of the plurality of glints to be clearly discernable.Examples of such shapes include, but are not limited to “U”, “Y” and“T”.

It will be appreciated that shape of the user's eye is spherical,therefore, the reflections of the structured light from a curved surfacemay get distorted. Thus, the structured light is projected on a region(for example, iris of the user's eye) of the user's eye where suchdistortion would be minimum.

Optionally, the plurality of micro-to-nano-sized components employrefractive optics. In such a case, the emitted light pulses (from the atleast one illuminator) are incident upon the plurality ofmicro-to-nano-sized components (for example, such as a plurality ofrefractive micro-lenses) that modify the structure of the light pulsespassing therethrough by way of refracting the light pulses. Optionally,different micro-to-nano-sized components can refract (namely, bend) theemitted light pulses at different angles. Optionally, in this regard,the plurality of micro-to-nano-sized components comprise a combinationof different components having different reflective indices.

Optionally, the plurality of micro-to-nano-sized components employdiffractive optics. In such as case, the emitted light pulses (from theat least one illuminator) are incident upon the plurality ofmicro-to-nano-sized components that modify the structure of the lightpulses passing therethrough by way of diffraction of the light pulses.Optionally, in this regard, the plurality of micro-to-nano-sizedcomponents constitute a diffraction grating. Notably, an arrangement ofthe plurality of micro-to-nano-sized components allows for diffractingthe emitted light pulses in a desired manner to produce the structuredlight.

Optionally, the plurality of micro-to-nano-sized components areimplemented by way of a foil comprising a plurality ofmicro-to-nano-sized particles dispersed therein. In such a case, thefoil is laminated onto the at least one first optical element.Optionally, in such a case, an adhesive is used for laminating the foilonto the at least one first optical element. Such a foil may betransparent or semi-transparent.

Optionally, the plurality of micro-to-nano-sized components areimplemented by way of plurality of fresnel structures. In such a case,the plurality of micro-to-nano-sized components are provided on asurface of the at least one first optical element to implement theplurality of fresnel structures. Notably, such a plurality of fresnelstructures effectively converts a single surface of the at least oneoptical element into multiple surfaces having same or different opticalcharacteristics. Optionally, in order to implement the plurality offresnel structures, the micro-to-nano structures are provided byroll-to-roll imprinting on the at least one first optical element. Themicro-to-nano structures could also be provided by employing techniquessuch as, but not limited to, electron beam lithography, direct writelaser lithography and diamond turning. It will be appreciated that suchthe plurality of fresnel structures are implemented on a surface of theat least one first optical element that lies on the optical path of theemitted light pulses. Furthermore, the plurality of fresnel structurescould employ refractive optics and/or diffractive optics, based upon adesired pattern of the structured light. Optionally, the plurality ofmicro-to-nano-sized components are positioned in close proximity to theuser's eye (for example, such as within a distance of less than 2.5 cmfrom the user's pupil).

In an example implementation, the at least one first optical element isimplemented by way of a transparent flexible foil. In such an example,the plurality of fresnel structures are implemented via roll-to-rollimprinting of the plurality of micro-to-nano-sized components on thetransparent flexible foil. It will be appreciated that the structuredlight is reflected from the user's eye, for example, from the cornea ofthe user's eye. In such an instance, the at least one camera is operableto capture the image of the reflections of the structured light. In oneimplementation, the at least one camera is positioned in an optical pathof the reflections of the structured light to capture the image thereof.In such a case, the at least one camera is positioned in a manner suchthat the user's view is not obstructed. In another implementation, theat least one first optical element is arranged on an optical path of thereflections of the structured light, namely between the user's eye andthe at least one camera. Optionally, in such an implementation, the atleast one first optical element is configured to pass the reflections ofthe structured light therethrough, towards the at least one camera.Alternatively, optionally, in such an implementation, the at least onefirst optical element is configured to direct (for example, byreflection, refraction, diffraction, or a combination thereof) thereflections of the structured light towards the at least one camera.

In operation, the at least one camera captures the image of reflectionsof the structured light from the user's eye. The image is representativeof the form and the position of the reflections on the image plane ofthe at least one camera. In such an instance, the image depictspositions and/or arrangement (namely, intergeometry) of the reflectionsof the structured light. In other words, the image depicts the positionsand/or arrangement of the plurality of glints formed on the outersurface of the user's eye.

It will be appreciated that the term “image plane of the at least onecamera” generally relates to a region of the at least one camera whereatthe reflections of the structured light are focused, to create theaforesaid image. In other words, the image plane of the at least onecamera is an imaging surface of the at least one camera, and lies withinthe at least one camera. Optionally, the image plane of the at least onecamera is implemented by way of at least one chip comprising a pluralityof photo-sensitive elements implemented thereon. Examples of the atleast one camera include, but are not limited to, a digital camera, ablack-and-white camera, a Red-Green-Blue (RGB) camera, and an Infra-Red(IR) camera. The form of the reflections and the position of thereflections of the structured light from the user's eye, depicted in thecaptured image, are used to determine a gaze direction of the user'seye. Optionally, such a captured image is also employed to determine ageometry (namely, shape and structure) of the user's eye. It will beappreciated that human eye has an irregular shape, such as a shape thatsubstantially deviates from a perfect sphere. Therefore, the structuredlight is reflected at different angles by different regions of theuser's eye.

Optionally, the reflections of the structured light, represented in thecaptured image, appear as the plurality of glints positioned within asubstantially circular region of the surface of the user's eye, theplurality of glints being arranged into at least two columns, each ofthe plurality of glints being symmetrical about only one axis. It willbe appreciated that the circular region of the user's eye substantiallycorresponds to a region comprising the iris of the user's eye and/or theuser's pupil. Such a structured arrangement of the plurality of glints(owing to the pattern of the structured light) allows for easilydistinguishing the plurality of glints from reflections of light fromambient sources. Beneficially, the plurality of glints are utilized as aframe of reference to detect the positioning of the user's pupil.Moreover, since the plurality of the glints are symmetrical about onlyone axis, the orientation of the plurality of glints can be accuratelyrecognized even upon rotation thereof, based upon their shape.

More optionally, the plurality of glints are in a form of six charactersthat are arranged into two columns of three characters each, thecharacters being substantially V-shaped. In one embodiment, a firstcolumn of three characters is arranged on a right region of the user'seye, whilst a second column is arranged on a left region user's eye. Inanother embodiment, a first column of three characters is arranged onthe left region of the user's eye, whilst a second column is arranged onthe right region user's eye. In yet another embodiment, the first columnof three characters is arranged on a top region of the user's eye,whilst a second column is arranged on a bottom region of the user's eye.In still another embodiment, the first column of three characters isarranged on the bottom region of the user's eye, whilst a second columnis arranged on the top region of the user's eye. It will be appreciatedsubstantially V-shaped characters are symmetrical about the only oneaxis; therefore, orientations of the V-shaped characters can beaccurately recognized even upon rotation thereof, based upon theirshape. In an example, the V-shaped characters may also be represented as“V”. It will be appreciated that when the V-shaped characters arerotated by 90 degrees in clockwise direction, the V-shaped charactersmay be represented as “<”. Similarly, when the V-shaped characters maybe rotated by 180 degrees and 270 degrees in clockwise direction, theV-shaped characters may be represented as “∧” and “>” respectively.Furthermore, when rotated by 360 degrees, the V-shaped characters arerepresented as “V” which is similar to the original shape of theV-shaped characters. Thus, the character “V” is symmetrical about onlyone axis.

Moreover, optionally, a middle character of a given column is to beoriented differently from other characters of the given column. In anexample, if a top character (namely, the first character) and a bottomcharacter (namely, the third character) of the given column arerepresented as “>”, the middle character (for example, a secondcharacter present between the first character and third character) maybe represented as “<”. It will be appreciated that such orientation ofthe middle character of the given column allows for preciselydetermining the positions of each of the top, the middle and the bottomcharacters of the given column. Therefore, such an arrangement of theplurality of glints could be employed to determine the gaze direction ofthe user in instances wherein at least one glint among the plurality ofglints is not visible (namely, is not depicted) in the captured image(for example, when the user's eyes are partially closed). As an example,the structured light may have a pattern such that the top characters ofthe two columns should be represented as “<” and “>”, the middlecharacters of the two columns should be represented as “>” and “<” andthe bottom characters of the two columns should be represented as “<”and “>” respectively in the captured image. In such an instance, if thecaptured image only depicts the middle and the bottom characters, thetop characters are determined to be occluded by the user's eye.

The processor is configured to control the at least one illuminator toilluminate the user's eye. In operation, the processor is configured tocontrol the at least one illuminator to produce the light pulses. Forexample, the processor may control at least one time instance and/or atleast one time duration during which the light pulses are to be emitted.

The processor is configured to receive the captured image depicting thereflections of the structured light from the at least one camera. Theprocessor is configured to process the captured image to determine theform and the position of the reflections of the structured light in thecaptured image. Optionally, the processor is configured to determine aposition and an orientation of user's pupil with respect to the form andthe position of the reflections of the structured light in the capturedimage to detect the gaze direction of the user. It will be appreciatedthat use of the structured light enables the processor to determine theposition and the orientation of the user's pupil with respect to theform and position of reflections of the structured light. For example,even when some of the reflections of the structured light are notvisible in the captured image, the visible reflections can still beutilized to allow determination of the user's gaze with high certaintybased upon the form and position of the visible reflections of thestructured light.

Furthermore, in an exemplary implementation wherein the plurality ofglints are in the form of six characters, if more than six charactersare observed in the captured image, the form and positions of thereflections of the structured light can be easily and accuratelydistinguished from the reflections of light from ambient light sources.Notably, the shapes and orientations of the plurality of glints areuniquely discernable. Therefore, it will be appreciated that suchdetermination of the user's gaze by employing the structured lightallows for reducing errors and enhancing gaze-detection accuracy.

Optionally, when processing the captured image, the processor isconfigured to identify a polygon defined by the plurality of glints, andto determine the position of the user's pupil with respect to thepolygon, so as to detect the gaze direction of the user. In such a case,optionally, the polygon is defined by corners of the plurality ofglints. In an example, if the plurality of glints are in a form of twocolumns of V-shaped characters, corners of the polygon are defined bycorner-most V-shaped characters. In another example, the polygon may bedefined by a line passing through all the plurality of glints.Optionally, the plurality of glints are substantially reflected from theiris of the user's eye and the polygon is therefore substantiallydefined around the user's pupil. In operation, the processor isconfigured to determine the position of the user's pupil with respect tothe polygon. In an example, the position of the user's pupil may be at acenter of the polygon. In such an example, the processor may process thecaptured image to detect that the user is gazing at a central region ofthe visual scene (depicted in the input image). In another example, theposition of the user's pupil may be at a right region of the polygon. Insuch an example, the processor may process the captured image to detectthat the user is gazing at a left side region of the visual scene(depicted in the input image).

As mentioned previously, the processor optionally processes the capturedimage depicting the reflections of the structured light to determine thegeometry of the user's eye. In such a case, angles of reflection of thestructured light may be different when the structured light is incidentupon different regions of the user's eye. Therefore, orientation and/orsize of the plurality of glints allow for identifying geometry of theircorresponding regions of the user's eye. In an example, the depictedplurality of glints may comprise six characters arranged into twocolumns of three characters each. Optionally, in such an example, themiddle characters of the two columns are oriented differently from othercharacters of the two columns. Furthermore, one column may be formed ona right region of the user's eye, while the other column may be formedon a left region of the user's eye. In such a case, a first character isformed on a top-right region of the user's eye, a second character isformed on a middle-right region of the user's eye, and a third characteris formed on a bottom-right region of the user's eye. Similarly, afourth character is formed on a top-left region of the user's eye, afifth character is formed on a middle-left region of the user's eye, anda sixth character is formed on a bottom-left region of the user's eye.It will be appreciated that the structured light incident upon themiddle-right region and the middle-left region of the user's eye will bereflected at an angle such that the form of the second and the fifthcharacters are substantially similar to their emitted form (within thestructured light). However, the structured light incident upon otherregions of the user's eye (such as, top and bottom portions) will bereflected at substantially greater angles as compared to the structuredlight incident upon the middle regions of the user's eye. In such aninstance, the form of the first, third, fourth and sixth characters issubstantially different from their emitted form (within the structuredlight). Optionally, in this regard, the processor is configured to storethe form and the position of the structured light as well as the formand the position (namely, relative position) of the plurality of glints.The processor compares the aforesaid forms and positions to identifysimilarities and/or differences therebetween. Consequently, theprocessor is configured to determine the geometry of the user's eye.

Optionally, the processor is configured to calibrate the gaze-trackingsystem by (i) determining an initial position of the head-mounteddisplay apparatus with respect to the user's eye, whilst recording theform and the position of the reflections of the structured light asrepresented by a first image captured by the at least one camera. Forexample, when the head-mounted display apparatus is worn by the user, acalibration sequence may be started. In such an instance, uponadjustment of the head-mounted display apparatus by the user accordingto requirements thereof (such as, in a comfortable position), the user'seye is illuminated by the at least one illuminator. Subsequently, thefirst image is captured by the at least one camera to determine theinitial position of the head-mounted display apparatus with respect tothe user's eye. Such a captured first image will be representative ofthe form and the position of the reflections of structured lightcorresponding to the initial position of the head-mounted displayapparatus with respect to the user's eye.

Furthermore, optionally, the processor is configured to calibrate thegaze-tracking system by (ii) storing information indicative of theinitial position with respect to the recorded form and position of thereflections. For example, the form and the position of the reflectionsas represented by the captured first image that is stored, optionally,in a memory unit communicably coupled to the processor. In anotherexample, the processor is operable to store numerical values associatedwith the form and the position of the reflections, such as numericalvalues of coordinates associated with the reflections as represented bythe captured first image.

Moreover, optionally, the processor is configured to calibrate thegaze-tracking system by (iii) determining a change in the position ofthe head-mounted display apparatus with respect to the user's eye, basedupon a change in the form and/or the position of the reflections asrepresented by a second image captured at a later time with respect tothe recorded form and position of the reflections. For example, inoperation, the head-mounted display apparatus may shift from the initialposition thereof on the user's head due to movement of the user's head.In such an instance, the processor is operable to control the at leastone camera to capture the second image representative of the form and/orthe position of the reflections due to such movement of the user's head.In one example, the processor is configured to control the at least onecamera to capture new images at regular intervals during operation, suchas, at every five seconds during operation of the head-mounted displayapparatus. Furthermore, the processor is operable to compare the formand positions of reflections in the new images with the initial positionof the form and position of the reflections (depicted in the firstimage) to determine changes in positions of the head-mounted displayapparatus, and subsequently, calibrate the gaze-tracking systemaccording to such changes.

Optionally, the processor is configured to indicate the user to lookstraight for calibrating the gaze-tracking system with respect to theuser's eye. The user may be indicated to look straight to capture thefirst image.

Optionally, the gaze-tracking system further comprises at least onesecond optical element that is substantially transparent for visiblelight, but is substantially reflective for infrared light. Optionally,the structured light is reflected from the user's eye towards the atleast one second optical element. The at least one second opticalelement is optically configured in a manner to allow the visible lightto pass therethrough substantially, whilst reflecting the infrared light(for example, such as the reflections of the structured light)substantially. Such an arrangement of the at least one second opticalelement facilitates altering the optical path of the reflections of thestructured light towards the at least one camera, whilst allowing thevisible light to pass therethrough. Therefore, unwanted visible light isnot captured by the at least one camera. Optionally, the at least onesecond optical element is implemented by way of at least one of: asemi-transparent mirror, a semi-transparent film, a prism, a polarizer,an optical waveguide.

Optionally, when the at least one first optical element is implementedas the transparent flexible film or the flexible transparent foil, theat least one first optical element is laminated onto the at least onesecond optical element.

Optionally, the head-mounted display apparatus comprises at least onedisplay for rendering the input image, the at least one second opticalelement being positioned on an optical path of a projection of therendered input image and on an optical path of the reflections of thestructured light. Therefore, the at least one second optical elementthat is substantially transparent for visible light, allows theprojection of the rendered input image emanating from the at least onedisplay to pass towards the user's eye, whilst allowing the reflectionsof the structured light to be reflected towards the at least one camera.Therefore, the reflections of the structured light are prevented frompassing towards the at least one display, whilst ensuring that theprojection of the rendered input image, having the visible wavelength,is projected onto the user's eye.

Optionally, the at least one display is implemented by way of at leastone of: a Liquid Crystal Display (LCD), a Light Emitting Diode(LED)-based display, an Organic LED (OLED)-based display, a microOLED-based display, and a Liquid Crystal on Silicon (LCoS)-baseddisplay.

Optionally, the at least one display is implemented by way of aprojection screen associated with at least one projector. Optionally, inthis regard, the at least one projector is implemented by way of atleast one of: a Liquid Crystal Display (LCD)-based projector, a LightEmitting Diode (LED)-based projector, an Organic LED (OLED)-basedprojector, a Liquid Crystal on Silicon (LCoS)-based projector, a DigitalLight Processing (DLP)-based projector, and a laser projector.

According to an embodiment, the term “input image” relates to arepresentation of a visual scene of a fully-virtual simulatedenvironment (for example, a virtual reality environment) to be displayedvia the head-mounted display apparatus.

According to another embodiment, the term “input image” relates to arepresentation of a visual scene depicting at least one virtual objectoverlaid on a real world image. Examples of the at least one virtualobject include, but are not limited to, a virtual navigation tool, avirtual gadget, a virtual message, a virtual entity, and a virtualmedia. In such an instance, the at least one virtual object overlaid onthe real world image constitutes a visual scene of a resultant simulatedenvironment (for example, an augmented reality environment). Notably,the term “real world image” relates to an image depicting actualsurroundings of the user whereat he/she is positioned. Optionally, thehead-mounted display apparatus comprises an imaging system to capturethe real world image. More optionally, the head-mounted displayapparatus further comprises at least one optical equipment (for example,such as a mirror, a lens, a prism, and the like) to implement aforesaidoverlaying operation and to project the resultant simulated environmentonto the user' eyes.

According to yet another embodiment, the term “input image” used hereinrelates to a visual scene depicting a pictorial representation (namely,a visual perception) of a subject. Examples of the subject include, butare not limited to, an object, a person, a map, a painting, a graphicaldiagram, and text. Optionally, the input image is a two-dimensionalrepresentation of the subject.

In an embodiment the head-mounted display apparatus is configured toreceive the input image from the memory unit communicably coupledthereto. The memory unit could be configured to store the input image ina suitable format including, but not limited to, Moving Pictures ExpertsGroup (MPEG), Joint Photographic Experts Group (JPEG), Tagged Image FileFormat (TIFF), Portable Network Graphics (PNG), Graphics InterchangeFormat (GIF), and Bitmap file format (BMP).

In another embodiment, the head-mounted display apparatus is configuredto receive the input image from the imaging system of the head-mounteddisplay apparatus. In such an instance, an image sensor of the imagingsystem is configured to capture the input image. As an example, theinput image may depict a coffee shop whereat the user is positioned.

It is to be understood that the input image may also be referred to as a“displayed image”, a “virtual reality image”, a “virtual object image”,a “subject image”.

Optionally, the at least one display comprises at least one focusdisplay for rendering a focus image and at least one context display forrendering a context image, wherein a projection of the rendered contextimage and a projection of the rendered focus image together form aprojection of the aforesaid input image. Optionally, in this regard, thehead-mounted display apparatus comprises at least one optical combinerfor optically combining the projection of the rendered context imagewith the projection of the rendered focus image to create the projectionof the aforesaid input image. Furthermore, in such a case, the inputimage comprises the context image and the focus image. Therefore, thecontext and focus images are rendered substantially simultaneously, inorder to collectively constitute the rendered input image. It will beappreciated that the context image relates to a wide image of the visualscene, to be rendered and projected via the head-mounted displayapparatus. Furthermore, the focus image relates to another imagedepicting a part (namely, a portion) of the visual scene, to be renderedand projected via the head-mounted display apparatus. Moreover, thefocus image is dimensionally smaller than the context image. Optionally,an angular width of the projection of the rendered context image rangesfrom 40 degrees to 220 degrees, whereas an angular width of theprojection of the rendered focus image ranges from 5 degrees to 60degrees.

Optionally, the at least one context display and/or the at least onefocus display are selected from the group consisting of: a LiquidCrystal Display (LCD), a Light Emitting Diode (LED)-based display, anOrganic LED (OLED)-based display, a micro OLED-based display, and aLiquid Crystal on Silicon (LCoS)-based display.

Throughout the present disclosure, the term “optical combiner” usedherein relates to equipment (for example, such as optical elements) foroptically combining the projection of the rendered context image and theprojection of the rendered focus image to constitute the projection ofthe input image. Beneficially, the at least one optical combiner couldbe configured to simulate active foveation of a human visual system. Inan embodiment, the at least one optical combiner is implemented by wayof the at least second optical element.

Optionally, the gaze-tracking system comprises an exit optical elementarranged on an optical path of the projection of the input image.Optionally, the exit optical element is configured to direct theprojection of the input image towards the user's eye, namely when thehead-mounted display apparatus is worn by the user. Optionally, the exitoptical element modifies an optical path and/or optical characteristicsof the projection of the input image prior to directing the projectionof the input image towards the user's eye. In one example, the exitoptical element may magnify a size (or angular dimensions) theprojection of the input image.

In one embodiment, the exit optical element is arranged between the atleast one display and the user's eye. In another embodiment, the exitoptical element is arranged between the at least one first opticalelement and the user's eye. It will be appreciated that relativeposition of the at least one first optical element is interchangeablewith the exit optical element.

Optionally, the exit optical element is implemented by way of at leastone of: a convex lens, a plano-convex lens, a Liquid Crystal (LC) lens,a liquid lens, aspherical lens, achromatic lens.

In an example implementation, the at least one first optical element isaffixed with the exit optical element (namely, an ocular lens of thehead-mounted display apparatus). In an example, the at least one firstoptical element is implemented as the transparent flexible film or theflexible transparent foil. In such an example, the at least one firstoptical element is laminated onto the exit optical element. It will beappreciated that the at least one first optical element may be laminatedonto a first side or a second side of the exit optical element, whereinthe first side is facing towards the user's eye and the second side isfacing towards the at least one display.

Optionally, the gaze-tracking system comprising at least one light guidefor guiding the light pulses emitting from the at least one illuminatortowards the at least one first optical element. Throughout the presentdisclosure, the term “light guide” used herein relates to an opticalelement that is operable to guide (namely, direct) the light pulsesemitted by the at least one illuminator towards the at least one firstoptical element. In one example, the light guide is associated with oneor more coupling elements for directing the light pulses emitted by theat least one illuminator into or out of the light guide. For example,the light guide may be associated with an inlet coupling element fordirecting light emitted by the at least one illuminator into the lightguide and an outlet coupling element for directing the emitted lightfrom the light guide towards the at least one first optical element.

In an exemplary implementation, the at least one illuminator isimplemented by way of an infrared light projector. In such an example,at least one infrared light projector may be arranged near the user'seye such that light pulses emitted by the at least one infrared lightprojector are incident upon the inlet coupling element of the lightguide. In such an instance, the light guide may be operable to directthe light pulses towards the outlet coupling element and subsequently,towards the at least one first optical element.

In another exemplary implementation, the at least one illuminator may beimplemented by way of two illuminators that are arranged at a peripheryof the exit optical element. In such a case, two light guides areprovided corresponding to the two illuminators such that the emittedlight pulses are directed from the two illuminators towards the inletcoupling elements of the two light guides, from the inlet couplingelements towards the outlet coupling elements of the two light guidesand subsequently, towards the at least one first optical element.

In yet another exemplary implementation, the at least one illuminator isimplemented by way of at least one display that is arranged at theperiphery of the exit optical element and is operable to produce thelight pulses in the form of an image and/or video. In such a case, theat least one first optical element is positioned in front of the exitoptical element. Furthermore, in this regard, the at least oneilluminator is implemented by way of at least one pixel of the at leastone display, wherein the at least one display is to be employed to flasha form towards the at least one first optical element to produce thestructured light. For example, such at least one display of thehead-mounted display apparatus may be the at least one focus display ofthe head-mounted display apparatus. In such an instance, the at leastone focus display is operable to flash the form to produce thestructured light having the predefined shape.

Optionally, the gaze-tracking system further comprises at least oneother illuminator for emitting light pulses to illuminate the user's eyefor enabling detection of the position of the user's pupil. It will beappreciated that wavelength of the light pulses emitted by the at leastone other illuminator is different from the wavelength of the lightpulses emitted by the at least one first illuminator. Optionally, the atleast one other illuminator emits the light pulses substantially towardsthe user's eye. It will be appreciated that the at least one otherilluminator is optionally arranged for illuminating the user's eye so asto facilitate both bright-pupil tracking and dark-pupil tracking.Furthermore, for implementing the bright-pupil tracking, the lightpulses emitted by the at least one other illuminator are arranged to beincident upon the user's eye substantially along the view direction ofthe user's eye. Moreover, for implementing the dark-pupil tracking, thelight pulses emitted by the at least one other illuminator are arrangedto be incident upon the user's eye substantially away from (namely,offset from) the view direction of the user's eye.

The present disclosure also relates to the method as described above.Various embodiments and variants disclosed above apply mutatis mutandisto the method. Optionally, in the method, the plurality ofmicro-to-nano-sized components employ refractive optics. Alternatively,optionally, in the method, the plurality of micro-to-nano-sizedcomponents employ diffractive optics.

Optionally, in the method, the reflections of the structured light,represented in the captured image, appear as the plurality of glintspositioned within the substantially circular region of the surface ofthe user's eye, the plurality of glints being arranged into the at leasttwo columns, each of the plurality of glints being symmetrical about theonly one axis. More optionally, in the method, the plurality of glintsare in the form of six characters that are arranged into two columns ofthree characters each, the characters being substantially V-shaped.Optionally, in this regard, the middle character of the given column isoriented differently from other characters of the given column.

Optionally, in the method, the processing of the captured imagecomprises: identifying the polygon defined by the plurality of glints;and determining the position of the user's pupil with respect to thepolygon, so as to detect the gaze direction of the user.

Optionally, in the method, the light pulses emitted by the at least oneilluminator have the infrared wavelength or the near-infraredwavelength.

DETAILED DESCRIPTION OF THE DRAWINGS

Referring to FIG. 1, illustrated is a block diagram of a gaze-trackingsystem 100 for use in a head-mounted display apparatus (not shown), inaccordance with an embodiment of the present disclosure. Thegaze-tracking system 100 comprises at least one illuminator 102 foremitting light pulses, at least one first optical element 104 comprisinga plurality of micro-to-nano-sized components, at least one camera 106for capturing an image of reflections of structured light from a user'seye and a processor 108 coupled in communication with the at least oneilluminator 102 and the at least one camera 106. The processor 108 isconfigured to control the at least one illuminator 102 and the at leastone camera 106, and to process the captured image to detect a gazedirection of the user.

Referring to FIGS. 2A-2F illustrated exemplary implementations of agaze-tracking system 200 in use within a head-mounted display apparatus,in accordance with various embodiments of the present disclosure. It maybe understood by a person skilled in the art that the FIGS. 2A-2Finclude simplified arrangements for implementation of the gaze-trackingsystem 200 for sake of clarity, which should not unduly limit the scopeof the claims herein. The person skilled in the art will recognize manyvariations, alternatives, and modifications of embodiments of thepresent disclosure.

As shown in FIGS. 2A-2F, the gaze-tracking system 200 comprises at leastone illuminator, depicted as an illuminator 202 for emitting lightpulses. For sake of simplicity, the emitted light pulses are depicted byray A. Furthermore, the gaze-tracking system 200 comprises at least onefirst optical element, depicted as a first optical element 204comprising a plurality of micro-to-nano-sized components. Such aplurality of micro-to-nano-sized components are depicted as components206. In operation, when the emitted light pulses A are incident upon theplurality of micro-to-nano-sized components 206, a structure of thelight pulses is modified to produce structured light. As shown, thestructured light is depicted by ray B. The produced structured light Bis to be used to illuminate a user's eye 208 when the head-mounteddisplay apparatus is worn by the user. Furthermore, the gaze-trackingsystem 200 comprises at least one camera, depicted as a camera 210 forcapturing an image of reflections of the structured light from theuser's eye 208. Such reflections of the structured light are depicted byray C.

In FIGS. 2B-2F, the gaze-tracking system 200 optionally comprises atleast one second optical element, depicted as a second optical element212. Such a second optical element 212 is optionally substantiallytransparent for visible light, but is substantially reflective forinfrared light (for example, such as the reflections of the structuredlight C). Moreover, the head-mounted display apparatus optionallycomprises at least one display, depicted as a display 214 for renderingan input image, the at least one second optical element 212 beingpositioned on an optical path of a projection of the rendered inputimage and on an optical path of the reflections of the structured lightC. Furthermore, as shown in the gaze-tracking system 200, an exitoptical element 216 is optionally positioned on an optical path of theproduced structured light B and the reflections of the structured lightC.

In FIG. 2C, the emitted light pulses A are incident upon the at leastone second optical element 212. The at least one second optical element212 is configured to reflect the emitted light pulses A towards the atleast one first optical element 204. The micro-to-nano-sized components206 allow for modifying the structure of the emitted light pulses A toproduce the structured light B.

In FIGS. 2D and 2E, the gaze-tracking system 200 optionally comprises atleast one light guide, depicted as a light guide 218 for guiding thelight pulses A emitting from the at least one illuminator 202 towardsthe at least one first optical element 204. Consequently, the emittedlight pulses A are incident upon the plurality of micro-to-nano-sizedcomponents 206. In FIG. 2D, the light guide 218 is placed obliquely(namely, at an angle) with respect to the at least one second opticalelement 212. However, in FIG. 2E, the light guide 218 is placedsubstantially parallely with respect to the at least one second opticalelement 212 and the at least one first optical element 204.

In FIG. 2F, the at least one illuminator 202 is implemented by way oftwo illuminators 202A and 202B. Furthermore, the at least one lightguide 218 is implemented as light guides 218A and 218B for guiding thelight pulses A emitting from the at least one illuminator 202A and 202B,towards the at least one first optical element 204A and 204Brespectively. As shown, the illuminators 202A and 202B, and the lightguides 218A and 218B, are provided near the exit optical element 216 ina manner such that the light guides 218A and 218B do not obstruct afield of view of the user.

Referring to FIG. 3, illustrated is an exemplary image of a user's eye300 captured by at least one camera of a gaze-tracking system, inaccordance with an embodiment of the present disclosure. As shown,reflections of the structured light, represented in the captured image,appear as a plurality of glints 302 positioned within a substantiallycircular region of a surface of the user's eye 300. The plurality ofglints 302 are arranged into at least two columns, each of the pluralityof glints being symmetrical about only one axis. As shown, the pluralityof glints 302 are optionally in a form of six characters, depicted ascharacters 304, 306, 308, 310, 312 and 314, that are arranged into twocolumns of three characters each, the characters being substantiallyV-shaped. Moreover, as shown, a middle character (for example, such asthe middle characters 306 and 312) of a given column is optionallyoriented differently from other characters of the given column.

Referring to FIGS. 4A and 4B, illustrated are zoomed-in views ofexemplary images of a user's eye depicting a plurality of glints 402therein, in accordance with different embodiments of the presentdisclosure. As shown, the plurality of glints 402 define a polygon 404.A processor (not shown) of the gaze-tracking system is configured todetermine a position of the user's pupil with respect to the polygon404, so as to detect the gaze direction of the user. Furthermore, asshown, the plurality of glints 402 are in a form of six characters thatare arranged into two columns of three characters each, the charactersbeing substantially V-shaped.

In FIG. 4A, the middle characters of the given columns are orienteddifferently from other characters of the given columns whereas in FIG.4B, the middle characters of the given columns are orientedsubstantially similarly to the other characters of the given columns.

Referring to FIG. 5, illustrated are steps of a method 500 of tracking auser's gaze, via a gaze-tracking system of a head-mounted displayapparatus, in accordance with an embodiment of the present disclosure.At step 502, structured light is produced via at least one illuminatorand at least one first optical element of the gaze-tracking system toilluminate a user's eye when the head-mounted display apparatus is wornby the user. The at least one first optical element comprises aplurality of micro-to-nano-sized components that are shaped and arrangedrelative to each other in a manner that, when incident upon theplurality of micro-to-nano-sized components, a structure of light pulsesemitted by the at least one illuminator is modified to produce thestructured light. At step 504, an image of reflections of the structuredlight from the user's eye is captured via at least one camera of thegaze-tracking system, wherein the image is representative of a form ofthe reflections and a position of the reflections on an image plane ofthe at least one camera. At step 506, the captured image is processed todetect a gaze direction of the user.

The steps 502 to 506 are only illustrative and other alternatives canalso be provided where one or more steps are added, one or more stepsare removed, or one or more steps are provided in a different sequencewithout departing from the scope of the claims herein.

Modifications to embodiments of the present disclosure described in theforegoing are possible without departing from the scope of the presentdisclosure as defined by the accompanying claims. Expressions such as“including”, “comprising”, “incorporating”, “have”, “is” used todescribe and claim the present disclosure are intended to be construedin a non-exclusive manner, namely allowing for items, components orelements not explicitly described also to be present. Reference to thesingular is also to be construed to relate to the plural.

What is claimed is:
 1. A gaze-tracking system for use in a head-mounteddisplay apparatus, the gaze-tracking system comprising: at least oneilluminator for emitting light pulses; at least one first opticalelement comprising a plurality of micro-to-nano-sized components, theplurality of micro-to-nano-sized components being shaped and arrangedrelative to each other in a manner that, when incident upon theplurality of micro-to-nano-sized components, a structure of the lightpulses is modified to produce structured light, wherein the producedstructured light is to be used to illuminate a user's eye when thehead-mounted display apparatus is worn by the user; at least one camerafor capturing an image of reflections of the structured light from theuser's eye, wherein the plurality of micro-to-nano-sized componentscause the reflections of the structured light to appear as a pluralityof glints, each of the plurality of glints being symmetrical about onlyone axis; wherein the plurality of glints are represented in thecaptured image as positioned within a circular region of a surface ofthe user's eye, the plurality of glints being arranged into at least twocolumns, and wherein the plurality of glints are in a form of sixcharacters that are arranged into two columns of three characters each,the characters being V-shaped; wherein the image is representative of aform of the reflections and a position of the reflections on an imageplane of the at least one camera; and a processor coupled incommunication with the at least one illuminator and the at least onecamera, wherein the processor is configured to control the at least oneilluminator and the at least one camera, and to process the capturedimage to detect a gaze direction of the user.
 2. The gaze-trackingsystem of claim 1, wherein the plurality of micro-to-nano-sizedcomponents employ refractive optics.
 3. The gaze-tracking system ofclaim 1, wherein the plurality of micro-to-nano-sized components employdiffractive optics.
 4. The gaze-tracking system of claim 1, wherein theplurality of micro-to-nano-sized components are implemented by way ofplurality of Fresnel structures.
 5. The gaze-tracking system of claim 1,wherein a middle character of a given column is to be orienteddifferently from other characters of the given column.
 6. Thegaze-tracking system of claim 1, wherein, when processing the capturedimage, the processor is configured to identify a polygon defined by theplurality of glints, and to determine a position of the user's pupilwith respect to the polygon, so as to detect the gaze direction of theuser.
 7. The gaze-tracking system of claim 1, further comprising atleast one light guide for guiding the light pulses emitting from the atleast one illuminator towards the at least one first optical element. 8.The gaze-tracking system of claim 1, wherein the light pulses emitted bythe at least one illuminator have an infrared wavelength or anear-infrared wavelength.
 9. The gaze-tracking system of claim 1,further comprising at least one second optical element that istransparent for visible light, but is reflective for infrared light. 10.The gaze-tracking system of claim 9, wherein the head-mounted displayapparatus comprises at least one display for rendering an input image,the at least one second optical element being positioned on an opticalpath of a projection of the rendered input image and on an optical pathof the reflections of the structured light.
 11. The gaze-tracking systemof claim 1, further comprising at least one other illuminator foremitting light pulses to illuminate the user's eye for enablingdetection of the position of the user's pupil.
 12. A method of trackinga user's gaze, via a gaze-tracking system of a head-mounted displayapparatus, the method comprising: producing structured light, via atleast one illuminator and at least one first optical element of thegaze-tracking system, to illuminate a user's eye when the head-mounteddisplay apparatus is worn by the user, wherein the at least one firstoptical element comprises a plurality of micro-to-nano-sized componentsthat are shaped and arranged relative to each other in a manner that,when incident upon the plurality of micro-to-nano-sized components, astructure of light pulses emitted by the at least one illuminator ismodified to produce the structured light; capturing an image ofreflections of the structured light from the user's eye, via at leastone camera of the gaze-tracking system, wherein the plurality ofmicro-to-nano-sized components cause the reflections of the structuredlight to appear as a plurality of glints, each of the plurality ofglints being symmetrical about only one axis, wherein the plurality ofglints are represented in the captured image as positioned within acircular region of a surface of the user's eye, the plurality of glintsbeing arranged into at least two columns, and wherein the plurality ofglints are in a form of six characters that are arranged into twocolumns of three characters each, the characters being V-shaped; whereinthe image is representative of a form of the reflections and a positionof the reflections on an image plane of the at least one camera; andprocessing the captured image to detect a gaze direction of the user.13. The method of claim 12, wherein the plurality of micro-to-nano-sizedcomponents employ refractive optics.
 14. The method of claim 12, whereinthe plurality of micro-to-nano-sized components employ diffractiveoptics.
 15. The method of claim 12, wherein a middle character of agiven column is oriented differently from other characters of the givencolumn.
 16. The method of claim 12, wherein the processing of thecaptured image comprises: identifying a polygon defined by the pluralityof glints; and determining a position of the user's pupil with respectto the polygon, so as to detect the gaze direction of the user.
 17. Themethod of claim 12, wherein the light pulses emitted by the at least oneilluminator.